Fully Nonparametric Probability Density Function Estimation with Finite Gaussian Mixture Models

Flexible and reliable probability density estimation is fundamental in unsupervised learning and classification. Finite Gaussian mixture models are commonly used to serve this purpose. However, they fail to estimate unknown probability density functions when used for nonparametric probability density estimation, as severe numerical difficulties may occur when the number of components increases. In this paper, we propose fully nonparametric density estimation by penalizing the covariance matrices of the mixture components according to the regularized Mahalanobis distance. As a consequence, the singularities in the loglikelihood function are avoided and the quality of the estimation models is significantly improved.